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Description 
BACKGROUND 

5 [0001] The present invention relates generally to imaging sensors, and more particularly, to a scene based nonuni- 
fbrmity correction method for use with such imaging sensors. 

[0002] Nonuniformities appear at an output display of an imaging sensor as fixed pattern noise. The nonuniformities 
are described as noise because they result in undesirable information. The nonuniformities are described as a fixed pat- 
tern because their characteristics do not change (or change relatively slowly) with time. These nonuniformities may also 

io be thought of as detector gain and offset errors. In the method of the present invention, all errors are treated as offset 
errors. Thus, the present invention accurately measures the detector offsets using actual scene information. 
[0003] Once the offset errors have been measured, there are several ways in which the corrections may be applied. 
They may be used as the only source of error correction. They may also be used as fine offset correction terms in con- 
junction with coarse offset terms and gain correction terms. These other terms may be calculated using a number of 

15 different methods. These methods include coarse offset terms calculated using a thermal reference source; coarse off- 
set and gain terms calculated as part of system initialization; and fine gain terms calculated using thermal reference 
sources or scene-based methods. 

[0004] EP-A-043991 2 discloses such a dynamic detector processing circuit for processing detector output data using 
detector-error-correcting offset coefficients. 

20 [0005] In one current method employed by the assignee of the present invention, one or more thermal reference 
sources are used to measure nonuniformities for a scanning infrared sensor and provide data for the calculation of cor- 
rection coefficients that are employed to correct for the nonuniformities. There are several disadvantages related to the 
use of a thermal reference source-based correction system First, there is added mechanical complexity which leads to 
increased system cost. Secondly, system performance may suffer. 

25 [0006] System performance suffers for two reasons, in many cases, a separate optical path is utilized for each thermal 
reference source. Thus, the correction coefficients calculated using the thermal reference source optical path may not 
be the proper ones for the optical path of the scanning infared sensor. This leads to imperfect correction. In less sophis- 
ticated systems, the temperature of the thermal reference source cannot be controlled. In this case, the thermal refer- 
ence source may not be at the same temperature as the scene that is viewed. The correction coefficients thus 

30 correspond to the wrong part of the detector response curve. This also leads to imperfect correction. The present 
method avoids these problems by using scene temperature information. Furthermore, the present invention does not 
degrade the scene in any manner. 

[0007] WO-A-90/09076 discloses an apparatus and a method for providing scene-based nonuniformity compensation 
of an imaging system comprising an array of detectors and a processor which implements nonuniformity compensation 
35 of the detectors using a feedback loop. The processor generates compensation terms which are subsequently com- 
bined in a summer with the output signals from the detector array. The method comprises median, antimedian and aver- 
age filters. The system makes use of high-pass filtered regions of a fixed shape not associated with the image content. 

SUMMARY OF THE INVENTION 

40 

[0008] As defined in claim 1 , the present scene-based nonuniformity correction method is used to eliminate image 
defects in an imaging sensor or video system, such as a scanning infrared sensor or pushbroom sensor, for example, 
resulting from nonuniformities caused by an detector (focal plane array) and detector readout, for example. The present 
invention detects, measures, and corrects for nonuniformities in the video output of a imaging sensor without degrading 
45 the image. A set of correction terms is calculated and applied to a video signal produced by the sensor using either a 
feedback or feedforward configuration. After the correction terms are applied, the resultant video signal is suitable for 
display or further processing. 

[0009] The video image generated by the imaging sensor is processed such that a vector representing an offset cor- 
rection term is formed, and this vector is initially set to zero. Each element in this vector represents a correction term for 
so a particular detector of the imaging sensor. The vector is applied to each pixel of the image by a processor as the pixels 
are read from the focal plane array of the sensor. 

[001 0] To measure the offset error, the image is separated into vertically oriented regions, each comprising a plurality 
of channels. The average of each channel within a region is computed and a set of region vectors is formed, such that 
there is one region vector for each region. Each region vector is then processed and regions whose edges are larger 
55 than a predefined threshold are detected and marked. Then, each region vector is further separated into sub-regions. 
The isolated sub-regions are high-pass filtered without regard to adjacent sub-regions. The high-pass filter may be in 
the form of a finite impulse response (FIR) filter or an anti-median filter, for example. The anti-median filter is a nonlinear 
high-pass filter that converges very rapidly. Finally, the correction terms for each vertical region vector are averaged 
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together, resulting in a single correction vector that is applied to the video signal to reduce the effect of the nonuniform- 
ities. 

[001 1 ] The correction terms calculated for each vertical region may also be applied individually to each the detectors. 
In this case, the correction terms corresponding to a region are applied as the detector scans the scene and views a 

s portion corresponding to that particular region. The correction terms are smoothed at region boundaries to eliminate 
noise due to boundary transitions. This second method is less sensitive to gain errors in the detector. 
[001 2] There are several advantages in using the present method. First the mechanical complexity of the imaging sys- 
tem is reduced, which leads to lower costs because fewer components are required and there is reduced labor required 
for manufacturing and testing. Secondly an imaging system incorporating the present invention provides better perform- 

w ance. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[001 3] The various features and advantages of the present invention may be more readily understood with reference 
is to the following detailed description taken in conjunction with the accompanying drawings, wherein like reference 
numerals designate like structural elements, and in which: 

Fig. 1 shows a block diagram of a generic imaging sensor-system incorporating a scene based nonuniformity cor- 
rection method in accordance with the principles of the present invention; 
20 Fig. 2 is a block diagram illustrating calculation of fine offset terms used in the scene based nonuniformity correc- 
tion method of the present invention that is employed in the imaging sensor system of Fig. 1 ; and 
Fig. 3 is a flow diagram illustrating the scene based nonuniformity correction method in accordance with the princi- 
ples of the present invention. 

25 DETAILED DESCRIPTION 

[001 4] In order to better understand the present method or algorithm, reference is made to Fig. 1 , which shows a block 
diagram of a generic scanning infrared sensor system 10, or imaging system 10, incorporating a scene based nonuni- 
formity correction method 40 in accordance with the principles of the present invention. The scanning infrared sensor 

30 system 10 is comprised of a detector 1 1 and its readout 12, and the readout is coupled to system electronics 13 that 
implements scene based nonuniformity correction method 40. The system electronics 13 implements correction logic 
that produces coarse and fine correction terms that are applied to the processed video signal. The correction logic 
includes two offset and gain pairs, comprising a coarse offset and gain pair, and a fine offset and gain pair. The coarse 
offset and gain pair is comprised of coarse offset level and coarse gain terms 16, 17 that may be calculated using a 

35 thermal reference source (internal or external) and pre-stored in a nonvolatile memory 28. The coarse offset level term 
16 may also be calculated using 8 thermal reference source which is updated continuously. 
[001 5] The fine offset and gain pair is comprised of fine offset level and fine gain terms 1 8, 1 9. The fine gain term 1 9 
may be set to unity, calculated using thermal reference sources, or calculated using a scene-based algorithm. First and 
second adders 21, 23 and first and second multipliers 22, 24 are employed to appropriately combine the coarse and 

40 fine level and gain terms 16, 17, 18, 19to produce a corrected video output signal. The present method 40 or algorithm 
is used to estimate the fine level correction terms 18 and is performed in a nonuniformity estimator 20. The output of 
the nonuniformity estimator 20 has a loop attenuation factor (k) 25 applied thereto and is coupled to a first input of a 
third adder 26. A second input of the third adder 26 is provided by the fine level term 18. As such the fine level term 18 
is updated with the output of the nonuniformity estimator 20 multiplied by the loop attenuation factor (k) 25. 

45 [001 6] The expressions and equations that are used in implementing the present algorithm or method 40 will now be 
descrfoed. The variables associated with the system 10 are as follows: 

x(m, n) = input, 

y(m, n) = output. 
so Lc(m) - coarse offset level term 16, 

Gc(m) = coarse gain term 17, 

Lfr(m, n) = fine offset level term 18. 

G F (m) - fine gain term 19, 

L (m. n) - error estimate, and 
55 k a loop attenuation factor 25, 

where 
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m = (0. M-1); M=number of detectors, and 
n = (0, N-1); N = samples/detector. 

[001 7] The system input and output are thus related by the equation: 

5 

y (m, n) = G F (m) {(G c (m) [x (m, n) + L c (m)] + L F -(m, n)}. 

[001 8] The tine level terms are recursively updated after each frame. Thus, 

10 L F (m, n ) = M m - n ) + k M m » n )- 

[0019] Fig. 2 is a block diagram illustrating calculation of fine offset level terms 1 8 used in the scene bad nonuniformity 
correction method 40 of the present invention that is employed in the scanning infrared sensor system 10 of Fig. 1 The 
following terms are defined and are used in implementing the method 40 of the present invention. 

15 

y\ (m) - horizontal average of a vertical region 
F hp « high pass filter operator, 
hp, (m) a high pass version y,, 
T = threshold operator, 
20 b| (m) o threshold version of yj, 
B = segment boundary operator, 
F, p a low pass filter operator, 
Ipi, (m) ■ low pass version of y,, and 
d (m) = vertical region correction term, 

25 

where 

I a {0, L-1); L a number of regions, and 
r = {0, R-1}: R = samples/region. 

30 

[0020] During an active field time, the scene based nonuniformity correction method 40 collects scene data and cal- 
culates the average within each region, for each line therein, illustrated in box 31 . This operation is equivalent to imple- 
menting the expression: 

35 0+i)R-i 

= J £ y(m,i). 

i=IR 



40 [0021] Y|(m) is thus comprised of several column vectors, or region vectors, y|, one for each region. These vectors are 
then high pass filtered (F hp ), illustrated in box 32, and thresholded (T), illustrated in box 33, to detect edges. The edges 
are marked as boundaries. Thus, 

hp,(m) = (F hp y l ) 

45 

b,(m) = (Th P| ). 

[0022] For locally high-pass filtering these vectors are then in a first step low pass filtered, illustrated in box 34, using 
the boundary information. Pixels marked as boundaries are ignored. This is denoted by the low pass filter operator, F !p 
so and the boundary operator, B. That is, 

lp,(m) = (F lp (By j )). 

[0023] In a second step, each region vector, yi (m), is subtracted from its low pass filtered version in an adder 35, 
55 producing the correction term for each region, c, -(m). That is 

c,(m) = lp,(m)-y l (m). 
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[0024] Finally, the correction terms are either applied individually for each region, or averaged together, wherein the 
boundary pixels are ignored. If they are averaged together, illustrated in box 36 the error estimate is calculated using 
the equation: 

5 1 L 

L(m, n) = l£ C| (m). 

«=o 

[0025] For the purposes of completeness, Fig. 3 is a flow diagram illustrating the scene based nonunifbrmity correc- 
io tion method 40 in accordance with the principles of the present invention. A video input signal is provided, indicated in 
step 41 , such as from the infrared sensor 1 1 derived from an image. The video input signal is processed such that a 
vector representing offset correction terms is formed, and this vector is initially set to zero. Each element in this vector 
represents a correction term for a particular detector of the scanning infrared sensor 1 1 . The vector is applied to each 
pixel of the image by the processor 13 as the pixels are read from the focal plane array 12. 
15 [0026] To measure the offset error, the image is separated into vertically oriented regions, each comprising a plurality 
of channels. The average of each channel within a region is computed, indicated in step 42, and a set of region vectors 
is formed, such that there is one region vector for each region. Each region vector is then globally high pass filtered, 
and edges larger than a predefined threshold are detected, indicated in step 43. and marked, indicated in step 44. 
Then, each region vector is further separated into sub-regions, indicated in step 45. The isolated sub-regions are high- 
20 pass filtered without regard to adjacent sub-regions, indicated in step 46, followed by a low-pass filtering step 47. In a 
first embodiment of the method 40, the correction terms for each vertical region vector are averaged together, resulting 
in a single correction vector, indicated in step 48. 

[0027] The correction terms calculated for each vertical region may also be applied individually to each the detectors. 
In this second embodiment, the offset level error in each region for each channel is calculated, indicated in step 49, and 
25 wherein the offset level error at boundary edges is undefined because boundary pixels are ignored. The correction 
terms corresponding to a region are applied as the detector 1 1 scans the scene and views a portion corresponding to 
that particular region. The correction terms are smoothed at region boundaries to eliminate noise due to boundary tran- 
sitions. This second method 40 is less sensitive to gain errors in the detector. 

30 Claims 

1. A scene based nonuniformity correction method (40) for use with a scanning infrared sensor comprising an array 
(11) of detectors, said method (40) comprising the steps of: 

35 providing (41) a video input (x(m, n)) derived from an image; 

processing the video input signal such that a vector representing offset correction terms is formed, wherein 
each element in the vector represents a correction term for a particular detector (m) of the scanning infrared 
sensor (11), and wherein the vector is initially set to zero, said processing further comprising the following 
steps: 

40 separating the image into vertically oriented regions, each region (I) comprising a plurality of detector chan- 

nels; 

computing (42) an average (yi(m)) of each detector channel within a region (I) and forming from the thus 
obtained averages a set of region vectors (yi), such that there is one region vector for each region; 
globally high-pass f fltering (F hp ; 32) each region vector whereby edges larger than a predefined threshold are 
45 detected (43), and marked (44); 

separating (45) each region vector into sub-region vectors demarcated by the edges; 
locally high-pass filtering (46) each sub-region vector; 

computing (48. 49) the offset correction terms for each region vector from said high-pass filtered sub-region 
vectors; and 

so applying the offset correction terms to the video signal. 

2. The method of Claim 1 wherein the step of computing (48, 49) the correction terms comprises the step of averaging 
(48) the correction terms for each region vector together to produce a single correction vector. 

55 3. The method of Claim 1 wherein the step of computing (48, 49) the correction terms comprises the step of calculat- 
ing (49) the offset correction term for each detector channel in each region, wherein the offset correction terms at 
the edges are ignored. 
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4. The method of Claim 1 , 2 or 3 wherein the step of locally high-pass filtering (46) comprises the step of anti-median 
filtering a sub-region vector. 

5. The method of Claim 1. 2 or 3 wherein the step of locally high-pass filtering (46) comprises the step of low-pass 
5 filtering a sub-region vector and subtracting the resultant vector from its corresponding original region vector (yl). 

6. The method of Claim 1 , 2 or 3 wherein the step of locally high-pass filtering (46) comprises the step of medjan fil- 
tering a sub-region vector and subtracting the resultant vector from its corresponding original region vector(y,). 

w Patentanspruche 

1. Ein Verfahren (40) zur szenenabhangigen Korrektur von UngleichmaBigkert zur Verwendung mit einem abtasten- 
den Infrarotsensor, der eine Gruppe (11) von Detektoren aufweist, wobei das Verfahren (40) die Schritte aufweist: 

is Bereitstellen (41) eines von einem Bild abgeleiteten Videoeingangssignals (x(m, n)); 

Verarbeiten des Videoeingangssignals, derart, daB ein Vektor gebildet wird, der Offsetkorrekturterme repra- 
serrtiert. wobei jedes Element in dem Vektor einen Korrekturterm fur einen bestimmten Detektor (m) des abta- 
stenden Infrarotsensors (11) reprasentiert, und wobei der Vektor anfangs auf Null gesetzt wird, wobei das 
20 Verarbeiten des weiteren die folgenden Schritte aufweist: 

Aufteilen des Bildes in vertikal orientierte Bereiche, wobei jeder Bereich (I) eine Vielzahl von Detektorkanal en 
aufweist; 

25 Berechnen (42) eines Mittelwertes (yi(m)) fur jeden Detektorkanal innerhalb eines Bereichs (I) und Bilden 

eines Satzes von Bereichsvektoren (yj) aus den so erhaltenen Mittelwerten, derart, daB es fQr jeden Bereich 
einen Bereichsvektor gibt; 

global HochpassfiHern (F hp ; 32) eines jeden Bereichsvektors, wodurch Kanten, die grOBer als eine vorbe- 
30 stimmte Schwelle sind, detektiert (43) und markiert (44) werden; 

Aufteilen (45) eines jeden Bereichsvektors in Unterbereichsvektoren. die durch die Kanten abgegrenzt wer- 
den; 

35 lokal Hochpassfiltern (46) eines jeden Unterbereichsvektors; 

Berechnen (48, 49) der Offsetkorrekturterme fur jeden Bereichsvektor aus den hochpassgef ifterten Unterbe- 
reichsvektoren; und 

40 Beaufschlagen des Videosignals mit den Offsetkorrekturtermen. 

2. Das Verfahren nach Anspruch 1 , worin der Schritt des Berechnens (48, 49) der Korrekturterme den Schritt des Mit- 
telns (48) der Korrekturterme fur jeden Bereichsvektor aufweist, urn einen einzelnen Korrekturvektor zu erzeugen. 

45 3. Das Verfahren nach Anspruch 1, worin der Schritt des Berechnens (48, 49) der Korrekturterme den Schritt des 
Berechnens (49) des Offsetkorrekturterms fur jeden Detektorkanal in jedem Bereich aufweist, wobei die Offsetkor- 
rekturterme an den Kanten ignoriert werden. 

4. Das Verfahren nach Anspruch 1 , 2 Oder 3, worin der Schritt des lokal Hochpassf ilterns (46) den Schritt des Anti- 
so medianf ilterns eines Unterbereichsvektors aufweist. 

5. Das Verfahren nach Anspruch 1 , 2 oder 3, worin der Schritt des lokal Hochpassf ilterns (46) den Schritt des Tief- 
passf ilterns eines Uriterbereichsvektors und Subtrahierens des sich ergebenden Vektors von seinem zugehOrigen 
Originalbereichsvektor (y ( ) aufweist. 

55 

6. Das Verfahren nach Anspruch 1 , 2 Oder 3, worin der Schritt des lokal Hochpassf ilterns (46) den Schritt des Medi- 
anfilterns eines Unterbereichsvektors und Subtrahierens des sich ergebenden Vektors von seinem zugehOrigen 
Originalbereichsvektor (yO aufweist. 
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Revendications 

1 . Procede de correction de defaut d'uniformite base sur une scene (40), pour I'utilisation avec un capteur infrarouge 
a balayage comprenant un reseau (1 1) de detecteurs, ce procede (40) comprenant les etapes suivantes : 

5 

on fournit (41) un signal d'entree video (X(m, n)) obtenu a partir d'une image ; 

on traite le signal d'entree video de facon a former un vecteur representant des termes de correction de deca- 
lage. chaque element dans le vecteur representant un terme de correction pour un detecteur (m) particulier du 
capteur infrarouge a balayage (1 1), et le vecteur §tant fixe inrtialement a zero ; ce traitement comprenant en 
10 outre les etapes suivantes : 

on separe llmage en regions orientees verticalement, chaque region (I) comprenant une pluralite de canaux 
de detecteur ; 

on calcule (42) une moyenne (yj(m)) de chaque canal de detecteur a I'interieur d'une region (I) 
et on forme a partir des moyennes ainsi obtenues un ensemble de vecteurs de region (yl) . de facon qu'il y ait 
is un vecteur de region pour chaque region ; 

on effectue un f iltrage passe-haut global (F hp ;32) de chaque vecteur de region, grace a quoi des bords supe- 
rieurs a un seuil predef ini sont detectes (43), et marques (44) ; 

on separe (45) chaque vecteur de region en vecteurs de sous-region delimites par les bords ; 
on effectue un f iltrage passe-haut local (46) de chaque vecteur de sous-region ; 
20 on calcule (48, 49) les termes de correction de decalage pour chaque vecteur de region a partir des vecteurs 

de sous-region f iltres en mode passe-haut ; et 
on applique les termes de correction de decalage au signal video. 

2. Procede selon la revendication 1 , dans lequel I'etape de calcul (48, 49) des termes de correction comprend I'etape 
25 consistant a faire la moyenne (48) de I'ensemble des termes de correction pour chaque vecteur de region, pour 

produire un seul vecteur de correction. 

3. Procede selon la revendication 1 , dans lequel I'etape de calcul (48, 49) des termes de correction comprend I'etape 
consistant a calculer (49) le terme de correction de decalage pour chaque canal de detecteur dans chaque region, 

30 en negligeant les termes de correction de decalage aux bords. 

4. Procede selon la revendication 1 , 2 ou 3. dans lequel I'etape de f iltrage passe-haut local (46) comprend I'etape de 
fittrage d'anti-mediane d'un vecteur de sous-region. 

35 5. Procede selon la revendication 1 , 2 ou 3, dans lequel I'etape de f iltrage passe-haut local (46) comprend I'etape con- 
sistant a appliquer un f iltrage passe-bas a un vecteur de sous-region, et a soustraire le vecteur resultant de son 
vecteur de region d'origine correspondant (yi). 

6. Procede selon la revendication 1 , 2 ou 3, dans lequel I'etape de f iltrage passe-haut local (46) comprend I'etape con- 
40 sistant a effectuer un f iltrage de mediane d\jn vecteur de sous-region et a soustraire le vecteur resultant de son 
vecteur de region d'origine correspondant (yi). 
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Fig- 3 
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PROVIDING A VIDEO INPUT SIGNAL 
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WITHIN EACH REGION, I, CALCULATE THE 
AVERAGE VALUE FOR EACH CHANNEL m 
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DETECT EDGES OF EACH REGION 



44 

I l 

MARK PIXELS WHOSE VALUES ARE OVER A THRESHOLD 
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USE MARKED PIXELS TO DIVIDE EACH REGION 
INTO INDIVIDUAL SUB-REGIONS 
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HIGH-PASS FILTER THE ISOLATED SUB-REGIONS 
WITHOUT REGARD TO ADJACENT SUB-REGIONS 
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LOW PASS FILTER EACH SUB-REGION AND 
IGNORE EDGES OF THE SUB-REGIONS 
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CALCULATE THE OFFSET 
LEVEL ERROR IN EACH 

REGION FOR EACH 
CHANNEL. WHEREIN THE 
OFFSET LEVEL ERROR AT 
EDGES IS UNDEFINED 
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CALCULATE THE OFFSET 
LEVEL ERROR IN EACH 
REGION FOR EACH 
CHANNEL BY AVERAGING 
ACROSS THE REGIONS 
AND IGNORING 
UNDEFINED VALUES 
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